See @olver86 page 244.
For functionals, the variational derivative plays the role of the gradient of functions of several variables.
Given a smooth function $f$, the gradient is a 1-form $df$ such that for a vector $V$
$$ df_x(V)=\lim_{\epsilon \to 0} \frac{f(x+\epsilon V)-f(x)}{\epsilon}=\frac{d}{d\epsilon}f(x+\epsilon V)|_{\epsilon=0}, $$that is, it tells us how much the function varies along the direction specified by $V$.
If we think of $x$ not like a finite dimensional vector $(x_1,\ldots,x_p)$ but like a function $x:\{1,\ldots,p\}\to \mathbb{R}$ we can generalize this to the case $x:\mathbb{R}\to \mathbb{R}$, $t\mapsto x(t)$, being now $f$ not a function but a functional.
The question is: for a functional $f$, is there any mathematical object $\delta f_x$ such that for a function $V(t)$ gives us the new function
$$ \delta f_x(V)=\lim_{\epsilon\to 0}\frac{f(x+\epsilon V)-f(x)}{\epsilon}=\frac{d}{d\epsilon}f(x+\epsilon V)|_{\epsilon=0} $$?
Observe that in the usual case of $x$ being a vector and $f$ being a function we can interpret $df$ as the gradient vector $\nabla f$ (assuming the standard inner product, see here for more information) which satisfies
$$ df_x(V)=\langle (\nabla f)_x, V \rangle=\frac{d}{d\epsilon}f(x+\epsilon V)|_{\epsilon=0}. $$We can replace $\langle-,-\rangle$ with the usual inner product $\int dt$ for the Hilbert space $L^2$, so we can alternatively require to our new object $\delta f_x$, being $f$ a functional again, to satisfy
$$ \int \delta f_x \cdot V dt=\frac{d}{d\epsilon}f(x+\epsilon V)|_{\epsilon=0} $$There are lots of technical details we are missing here, but this is the idea.
Definition (@olver86 page 245)
Let $J[u]$ be a variational problem (for functions $u:\mathbb{R}^p \to \mathbb{R}^q$). The variational derivative of $J$ is the unique $q$-tuple
$$ \delta J[u]=(\delta_1 J[u],\ldots,\delta_q J[u]) $$such that given functions $f,\eta:\Omega \subset \mathbb{R}^p \to \mathbb{R}^q$, $\eta$ with compact support, satisfies:
$$ \frac{d}{d\epsilon}J[f+\epsilon \eta]|_{\epsilon=0}=\int_{\Omega}\delta J[f(x)]\cdot \eta(x)dx. \tag{1} $$$\blacksquare$
Remarks
a) To my knowledge, it doesn't have to exist...
b) I think that the quantity (1) should also be denoted by
$$ \delta J_f(\eta) $$to agree with $df_x(V)$. In this sense $\delta J$ would be something similar to differential of a function. The expression $\delta J_f=\delta J[f(x)]$ can be thought as another function of $x$ or, alternatively, as something eating functions $\eta$ and turning back numbers, i.e., something analogous to the 1-form $df_x$ in the usual case of functions of $\mathbb{R}^n$. Something like the duality between the gradient $\nabla f_x$ and $df_x$ (vectors correspond to functions).
c) Since the functional $J$ is coming from a variational problem we can compute an explicit formula for its variational derivative. Let's focus in the case of functions $u:\mathbb{R}\to \mathbb{R}$:
$$ \int_{\Omega}\delta J[f(x)]\cdot \eta(x)dx=\frac{d}{d\epsilon}J[f+\epsilon \eta]|_{\epsilon=0}= $$ $$ =\frac{d}{d\epsilon}|_{\epsilon=0} \int_{\Omega}L(x,pr^n(f+\epsilon \eta))dx= $$ $$ =\int_{\Omega}\dfrac{\partial L}{\partial u}\cdot \eta+\dfrac{\partial L}{\partial u_1}\cdot \eta^{\prime}+\cdots+\dfrac{\partial L}{\partial u_n}\cdot \eta^{n)}dx $$being $\eta^{j)}$ the $j$th derivative of $\eta$.
We can apply integration by parts, leaving apart the first term, and obtain that the expression above equals
$$ \int_{\Omega}\dfrac{\partial L}{\partial u}\cdot \eta dx+\int_{\Omega}-D_x\dfrac{\partial L}{\partial u_1}\cdot \eta dx+\cdots+\int_{\Omega}-D_x \dfrac{\partial L}{\partial u_{n}}\cdot \eta^{n-1)}dx $$and after $n$ steps:
$$ =\int_{\Omega}\dfrac{\partial L}{\partial u}\cdot \eta dx+\int_{\Omega}-D_x\dfrac{\partial L}{\partial u_1}\cdot \eta dx+\cdots+\int_{\Omega}\left(-D_x\right)^n \dfrac{\partial L}{\partial u_{n}}\cdot \eta dx= $$ $$ =\int_{\Omega} \sum_{j=0}^n \left(-D_x\right)^j \dfrac{\partial L}{\partial u_{j}}\cdot \eta dx $$So it should be
$$ \delta J[f(x)]=\sum_{j=0}^n \left(-D_x\right)^j \dfrac{\partial L}{\partial u_{j}} $$This last expression is called the Euler operator.
$\blacksquare$
Proposition (@olver86 page 246)
If $f(x)$ is an extrema of $J[u]$ then $\delta J[f(x)] \equiv 0$, the null function on $\Omega$.
$\blacksquare$
The Euler-Lagrange equations then appear from this proposition together with Remark c) above.
Given a functional $F:\mathcal{H}\mapsto \mathbb R$, being $\mathcal{H}$ a Hilbert space of functions, the associated gradient flow is given by the equation
$$ \frac{\partial \rho}{\partial t} = - \frac{\partial F}{\partial \rho}. \tag{1} $$for $\rho(t) \in \mathcal{H}$.
In other words, $\rho$ decreases along the gradient of $F$. The terminology stems from the 'finite dimensional' case, where a function $f(x,y,z)$ produces a vector field $V = \nabla f$, which is called its 'gradient vector field'. Then, as with any vector field, one can study the flow induced by that vector field, i.e. the flow of the dynamical system given by $\dot{x} = V(x)$.
In $(1)$, the notation $\frac{\partial F}{\partial \rho}$ denotes the so-called functional derivative of $F$ to $\rho$, which generalises the 'gradient' notion for functions. There exist multiple versions of the functional derivative, mainly because its definition depends on the function space on which $F$ acts. Anyway, the idea is to perturb $\rho$ a bit, i.e. to substitute $\rho \to \rho + \epsilon \phi$ with $0 < \epsilon \ll 1$, and work out the resulting expression.
Let $\mathcal{H}=L^2(\mathbb R^n)$, and $F$ such that for $u\in L^2(\mathbb R^n)$:
$$ F(u)=\frac{1}{2} \int|\nabla u(x)|^{2} d x $$That is, $F$ is the Dirichlet energy. Then the Heat Equation
$$ \partial_{t} u=\nabla^{2} u $$is the associated gradient flow problem.
________________________________________
________________________________________
________________________________________
Author of the notes: Antonio J. Pan-Collantes
INDEX: